A New Big Data Feature Selection Approach for Text Classification

Feature selection (FS) is a fundamental task for text classification problems. Text feature aims to represent documents using the most relevant features. This process can reduce size of datasets and improve performance machine learning algorithms. Many researchers have focused on elaborating efficient FS techniques. However, proposed approaches are evaluated small validated single machines. As textual data dimensionality becomes higher, traditional methods must be improved parallelized handle big data. paper proposes distributed approach based mutual information (MI) method, which widely applied in pattern recognition learning. A drawback MI that it ignores frequency terms during The proposal introduces namely, Maximum Term Frequency-Mutual Information (MTF-MI), term techniques quality selected implemented Hadoop MapReduce programming model. effectiveness MTF-MI demonstrated through several experiments multinomial Naïve Bayes classifier three datasets. Through series tests, results reveal method improves compared with four state-of-the-art macro-F1 micro-F1 measures.